Industry Led Use-Case Development for Human-Swarm Operations

In the domain of unmanned vehicles, autonomous robotic swarms promise to deliver increased efficiency and collective autonomy. How these swarms will operate in the future, and what communication requirements and operational boundaries will arise are yet to be sufficiently defined. A workshop was conducted with 11 professional unmanned-vehicle operators and designers with the objective of identifying use-cases for developing and testing robotic swarms. Three scenarios were defined by experts and were then compiled to produce a single use case outlining the scenario, objectives, agents, communication requirements and stages of operation when collaborating with highly autonomous swarms. Our compiled use case is intended for researchers, designers, and manufacturers alike to test and tailor their design pipeline to accommodate for some of the key issues in human-swarm ininteraction. Examples of application include informing simulation development, forming the basis of further design workshops, and identifying trust issues that may arise between human operators and the swarm.Comment: Accepted at AAAI 2022 Spring Symposium Series (Putting AI in the Critical Loop: Assured Trust and Autonomy in Human-Machine Teams

Identifying interaction types and functionality for automated vehicle virtual assistants: An exploratory study using speech acts cluster analysis

Onboard virtual assistants with the ability to converse with users are gaining favour in supporting effective human-machine interaction to meet safe standards of operation in automated vehicles (AVs). Previous studies have highlighted the need to communicate situation information to effectively support the transfer of control and responsibility of the driving task. This study explores ‘interaction types’ used for this complex human-machine transaction, by analysing how situation information is conveyed and reciprocated during a transfer of control scenario. Two human drivers alternated control in a bespoke, dual controlled driving simulator with the transfer of control being entirely reliant on verbal communication. Handover dialogues were coded based on speech-act classifications, and a cluster analysis was conducted. Four interaction types were identified for both virtual assistants (i.e., agent handing over control) - Supervisor, Information Desk, Interrogator and Converser, and drivers (i.e., agent taking control) - Coordinator, Perceiver, Inquirer and Silent Receiver. Each interaction type provides a framework of characteristics that can be used to define driver requirements and implemented in the design of future virtual assistants to support the driver in maintaining and rebuilding timely situation awareness, whilst ensuring a positive user experience. This study also provides additional insight into the role of dialogue turns and takeover time and provides recommendations for future virtual assistant designs in AVs

Identified handover tools and techniques in high-risk domains: using distributed situation awareness theory to inform current practices

In high-risk domains, poor shift handover has been identified as a key contributing factor in many incidents. This raises the question of: how can personnel collaborate effectively during shift changes? The vast majority of handover literature relates mainly to healthcare, aviation, energy generation and distribution domains. This review identified 19 distinct handover tools/techniques (HTTs) that domains employ to improve handover communication. The most prevalent HTT is standardisation in the form of a structured checklist, followed by the bidirectional exchange of information. This review assesses and summarises HTTs using ‘distributed situation awareness’ theory, and provides a comprehensive review on what is currently practised in high-risk domain handover, as well as a discussion around their potential contribution to raising distributed situation awareness.</p

Understanding the Impact of Induced Stress on Team Coordination Strategy in Multi-User Environments

Managing air traffic control, medical emergencies, and multi-robot systems are prime cases where human teams have to work on complex tasks. In such cases, these teams are continuously working under stress induced by instability, complexity, and time pressure. The success of such teams is primarily driven by effective team coordination. The objective of this study is to understand the impact of induced stress on human team coordination strategy. In this study, two online tasks were designed to induce stress in participants, one in single-user and the other in multi-user collaborative environments, measuring individual and collaborative teamwork performances respectively. Both experiments were conducted under induced time pressure and auditory distraction. Our analysis showed that team members prefer to switch between different strategies and thus, the coordination shifts from explicit to implicit coordination. However, in the single-user environment, participants' performance was influenced by their competitor’s performance, regardless of the participant’s abilities. Future research will determine how these effects are associated with physiological signals

Directability, eye-gaze, and the usage of visual displays during an automated vehicle handover task

The proposed next step in semi-automated vehicle technology is to allow a driver to conduct secondary tasks whilst automation is in full control of the driving task. The driver may be required to take control and promptly re-enter ‘the control loop’ if an automated feature reaches a design, geographical or capability boundary. In these circumstances it is of importance to raise the ‘Situation Awareness’ (SA) of the system through transactions between driver and vehicle. Communication literature suggests that a useful method for facilitating interactions between driver and automation is ‘directability’, guidance towards future actions or relevant pieces of information (e.g. road hazards). It is proposed that this would lead to improved SA. This study evaluated the role of directability in semi-automated vehicles by testing two research questions, how well can vocal communication from an automated assistant guide driver visual gaze, and how do drivers utilise visual displays during handover and manual driving? Participants took part in a simulated driving handover task on a highway. It was found that vocal guidance was effective in directing visual gaze. Further, the majority of visual-gaze in both handover and manual tasks was directed towards the road environment, and displays close to the road-view. This study provides additional evidence that vocal communication could serve as a reliable SA raising method, as well as provide insights into how different visual displays can be utilised for raising SA in level 3 and 4 automated vehicles

Conditionally and highly automated vehicle handover: a study exploring vocal communication between two drivers

Automated Vehicles with levels 3 and 4 capability involve the handover of control and responsibility between driver and automation. The handover task represents a vulnerability in a given system due to the reduction of situation awareness and possible breakdowns in communication. Handover assistants are a design approach proposed to counteract these vulnerabilities. This study investigated the concept of a vocal handover assistant by exploring information transferred, and the methods for doing so, in naturalistic vocal handover between two drivers. Additionally, it was hypothesised that scripted vocal methods would differ in measures of workload, usability, acceptance and the effect on longitudinal/lateral driving behaviour. In each trial, two drivers took part in a driving simulation exchanging control from one-another. Drivers took part in six conditions: four pre-set conditions related to a different interaction style and two ‘free-form’ conditions before and after pre-set conditions. Our results show a change in information-types transferred and methods adopted for communication from before to after taking part in pre-set conditions. Other findings highlight considerations to be made such as training, personalization, the transmission of priority as well as contextual information, and how handover methods may affect the control of the vehicle following handover. Grice's Maxims were applied to handover methods to facilitate discussion. We present four considerations for future design: efficiency, personalization, and presentation of prioritised and context-related information.</p

Festschrift in honour of Professor Neville Stanton: A lone crusader in a world of driving simulators

The automotive future has always pointed to a world of intelligent co-pilots and robot cars, but perhaps no more so than Knight Rider. In this 1980's television series the fictional Knight Industries Two Thousand (KITT) was a supercomputer on wheels with 1000 megabytes of memory. The protagonist was Michael Knight, a young loner on a crusade to champion the cause of the innocent and the helpless. This was a shadowy flight into the trials and tribulations of different levels of automation, re-claiming control when automation failed, and a wilful, chatty computer co-driver. An amusing metaphor, perhaps, for the research impact made by Neville Stanton in the field of vehicle automation. Without question – to paraphrase the Knight Rider outro – “one man can make a difference”. This festschrift in Neville's honour tells the story of how.</p

The effect of data visualisation quality and task density on human-swarm interaction

Despite the advantages of having robot swarms, human supervision is required for real-world applications. The performance of the human-swarm system depends on several factors including the data availability for the human operators. In this paper, we study the human factors aspect of the human-swarm interaction and investigate how having access to high-quality data can affect the performance of the human-swarm system - the number of tasks completed and the human trust level in operation. We designed an experiment where a human operator is tasked to operate a swarm to identify casualties in an area within a given time period. One group of operators had the option to request high-quality pictures while the other group had to base their decision on the available low-quality images. We performed a user study with 120 participants and recorded their success rate (directly logged via the simulation platform) as well as their workload and trust level (measured through a questionnaire after completing a human-swarm scenario). The findings from our study indicated that the group granted access to high-quality data exhibited an increased workload and placed greater trust in the swarm, thus confirming our initial hypothesis. However, we also found that the number of accurately identified casualties did not significantly vary between the two groups, suggesting that data quality had no impact on the successful completion of tasks

Industry led use-case development for human-swarm operations

In the domain of unmanned vehicles, autonomous robotic swarms promise to deliver increased efficiency and collective autonomy. How these swarms will operate in the future, and what communication requirements and operational boundaries will arise are yet to be sufficiently defined. A workshop was conducted with 11 professional unmanned-vehicle operators and designers with the objective of identifying use-cases for developing and testing robotic swarms. Three scenarios were defined by experts and were then compiled to produce a single use case outlining the scenario, objectives, agents, communication requirements and stages of operation when collaborating with highly autonomous swarms. Our compiled use case is intended for researchers, designers, and manufacturers alike to test and tailor their design pipeline to accommodate for some of the key issues in human-swarm ininteraction. Examples of application include informing simulation development, forming the basis of further design workshops, and identifying trust issues that may arise between human operators and the swarm

Identifying interaction types and functionality for automated vehicle virtual assistants: An exploratory study using speech acts cluster analysis

Onboard virtual assistants with the ability to converse with users are gaining favour in supporting effective human-machine interaction to meet safe standards of operation in automated vehicles (AVs). Previous studies have highlighted the need to communicate situation information to effectively support the transfer of control and responsibility of the driving task. This study explores ‘interaction types’ used for this complex human-machine transaction, by analysing how situation information is conveyed and reciprocated during a transfer of control scenario. Two human drivers alternated control in a bespoke, dual controlled driving simulator with the transfer of control being entirely reliant on verbal communication. Handover dialogues were coded based on speech-act classifications, and a cluster analysis was conducted. Four interaction types were identified for both virtual assistants (i.e., agent handing over control) - Supervisor, Information Desk, Interrogator and Converser, and drivers (i.e., agent taking control) - Coordinator, Perceiver, Inquirer and Silent Receiver. Each interaction type provides a framework of characteristics that can be used to define driver requirements and implemented in the design of future virtual assistants to support the driver in maintaining and rebuilding timely situation awareness, whilst ensuring a positive user experience. This study also provides additional insight into the role of dialogue turns and takeover time and provides recommendations for future virtual assistant designs in AVs. </p